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ABSTRACT 

Image segmentation (IS) is often the first and most important step in medical image analysis. Among other 
uses, it helps us perform the following tasks in images: to measure and visualize the anatomical structures of the brain; 
to highlight brain structural changes; and to delineate regions with severe pathology. Thus, it becomes much more 
significant in image-guided surgical planning. To date, a considerable body of research has burgeoned, which focuses 
on improving segmentation accuracy and reducing complexity. In the current study, we review well-selected articles 
describing the most popular segmentation methods commonly used in MRI studies. We mainly focus on highlighting the 
differences between these available methods, review their capabilities and advantages, and discuss their limitations and 
challenges. Conclusively, the study provides guidelines and directions for the appropriate segmentation approach(s) 
reliable and succinctly suite for MRI brain Images with good prospects. In order to fully understand concepts and 
problems behind segmentation methods, a need of thorough elaboration of segmentation perception is noteworthy. To do 
so, we have extensively established the most augmented introduction of basic segmentation concepts, and deeply 
explained the fundamental differences of brain-MRI preprocessing pipelines. 
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INTRODUCTION 

Segmentation of MRI-brain scans have played a key role in many studies in recent years. The main 
purpose of brain segmentation is localizing each unique brain structure based on tissue characteristics. With a 
fully-segmented brain, morphological information such as volume and shape, can be captured and used for future 
statistical analysis. The information helps researchers investigate how the brain is vulnerably affected by a 
neurological disease (see figure. 1). Most methods have previously been developed to solve the problem of brain 
segmentation. 

The intensity in MRI data offers a fundamental information in the segmentation process in most of these 
methods. Despite the intensity information being useful, the resulting segmentation is inaccurate due to noise, non¬ 
uniformity of intensity and poor image contrast in the MR data. Although MRI-data preprocessing pipeline 
involves processes that aim at reducing noise and artefacts, there is no guarantee that the structure and tissue 
intensities are unique as claimed priority. On the other hand, structures such as hippocampus and amygdala, for 
example, belong to gray matter, with almost identical intensities which makes difficult to differentiate them using 
the intensity aspect alone[l]. 
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Figure 1: Segmentation in Different Purposes: Delineate Multiple Sclerosis, 
Tumors, Cancers and Brain Regions Partitioning 


Consequently, brain segmentation accuracy would not be sufficient enough to perform meaningful statistical 


analysis. 


To increase the accuracy and robustness, algorithms have been developed to not only use the brain intensity 
information but also incorporate spatial and contextual information. The spatial pattern characteristic of the brain structure 
is well known even prior to MRI scanning. The information can easily enable locating amygdala as always anterior and 
superior to hippocampus. Studies have also identified that brain tissues often appear in clusters matter voxels, for instance, 
are generally surrounded by more gray matter voxels. Using this information seems to confidently be possible to 
incorporate spatial and contextual information for improved segmentation [2]. 


One among the methods that takes advantage of this direction is graphical models. In graphical models, the 
probability of each voxel-label given the current estimate of the labels in a finite number of neighboring voxels is 
computed while tuning various hyper parameters. Other techniques rely on tissue probability maps and manually 
segmented structural atlases of the particular image. 


MRI-Image Preprocessing Concepts 

Automated analysis of MR images is challenging due to intensity in homogeneity, variability of the intensity 
ranges and contrast, and noise. Therefore, prior to automated analysis, certain steps are required to make the images appear 
more similar, and these steps are commonly referred to as preprocessing. 

Typical preprocessing steps for structural brain MRI include the following key steps. 
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Figure 2: Segmentation Pipeline 


Image Registration Perspective 

Traditionally, the preprocessing begins with image registration. In image registration, two or more images with 
similar contents taken in regular interval (TR) or different viewpoints are overlaid (spatially aligned). More information 
about the patients’ health can be thoroughly extracted with multi-modal image registration, i.e., MRI, CT, PET, and 
SPECT. The obtained information plays a decisive role in describing the extent of pathology. The most common type of 
registration in preprocessing pipeline is co registration sometimes referred to intra-subject registration. Under this, single 
subjects images acquired from two or more modalities are spatially realigned (see figure 2) 

The more specific and practical scenario is co-registration involving anatomical image, i.e., T1 weighted images, 
and functional (fMRI) images with the purpose of correcting for motion. Its use can be extended to spatial space standard 
normalization which is a potential step in the MRI cross-sectional study [3] 

SEGMENTATION METHODS 

Diverse applications in medical imaging have led to the development of various segmentation techniques. Many 
have claimed that no single method can be suitable for all types of images nor are all methods equally good for a particular 
type of image processing pipeline. However, it has recently been vindicated that a single approach designed for a certain 
images class could be easily extended to another class. Therefore, we opt to explore in depth these available segmentation 
methods and critically analyze their limitations to help suggest an appropriate direction for the stable segmentation method. 
Our hypothesis stands to argue that the feasible segmentation method could be available or developed that would be 
adjustable to fit various preprocessing pipelines for divergent applications [4]. 
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CNN based -Segmentation perception 



Figure 3: CNN Based-Segmentation 
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Figure 4: Broader View of Deep Learning 


Deformation-Based Morphometry 


Deformation-based morphometry (DBM) gained confidence and popularity through its efficiency in identifying 
global and gross brain volume changes. It is capable of characterizing volumes and shapes of the brain across different 
population. The DBM heavily relies on deformation generated by nonlinear registration with a brain template. 


The resulting deformation field contains useful information about local volume and shape of the patients’ brain. 
To recover this information, the Jacobian determinant, also known as the volume expansion factor, is applied to the 
deformation field. The Jacobian, in theory, describes the amount of stretching, rotating and shearing that the non-linear 
transformation has imposed on MR scan. With the determinant applied to the Jacobian, local brain growth or shrinkage, 
relative to the brain template, is revealed. In particular, the Jacobian determinant greater than one (JD> 1) signifies 
volumetric expansion, while that less than one (JD<1) is associated with local shrinkage. When the value is one (JD=1), it 
suggests that there is no volume change. 


When many subjects scans are aligned to a common anatomical space, statistical models can be applied across the 
Jacobian determinant of the deformation fields, on a voxel-wise basis, to identify localized group volumetric 
differences [6, 7]. 
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Voxel-Based Morphometry 

In contrast to DBM, voxel-based morphometry (VBM) detects regional brain volume changes at the mesoscopic 
scale, i.e., several millimeters. Identification of regional tissue volume differences of gray matter, white matter, and CSF is 
absolutely achieved in VBM. It is desirable to use DBM and VBM together because they complement each other by 
capturing brain volume changes at both the macroscopic and mesoscopic scales. 

Initially, MR scans are linearly transformed into a common anatomical space. This is followed by tissue 
classification, where each voxel will be assigned one of four classes: gray matter, white matter, CSF, or non-brain tissue. 
The resulting tissue maps are then blurred with a Gaussian kernel. Blurring is an important stage in VBM because it helps 
facilitate non-linear registration, performed later, by providing a better capture range for local displacements and mitigating 
registration errors. Blurring also permits Gaussian assumptions to be applied in later statistical analysis. If these tissue 
maps were not blurred, then complex discrete statistics would be required. The resulting blurred tissue maps are referred to 
as tissue density maps, which reflect the local concentration of each respective tissue. 

Finally, each tissue density map is non-linearly registered its corresponding tissue probability map, to account for 
global brain shape differences. Statistical analysis can then be performed on a voxel wise basis across the entire population 
to determine regional tissue volume differences. VBM can also incorporate a modulation step where the Jacobian 
determinant of the deformation field from the estimated non-linear transformation are multiplied with each voxel. By doing 
so, it will preserve the volume of particular tissues that may have been expanded or contracted during non-linear 
registration. When the modulation step is included the technique is referred to as optimized VB [8, 9, 10]. 

Multi-Atlas Registration Based Method 

Generally segmentation methods rely on intensity information and higher order structural information of objects. 
The prominent example is Multi-Atlas registration method, which is based on a specific set of templates whereby in a 
medical application, typically contains labelling of anatomical structures. 

The field of brain segmentation is dominated by multi-atlas based methods. They are driven by a combination of 
spatial normalization through registration followed by a classification step, which can be a simple majority voting. If an 
atlas or templates of the human brain for a specific population of interest are available, then atlas-based method can be a 
powerful tool for brain MRI segmentation. The main advantage of these methods is the possibility to segment any brain 
structure available in the atlas without any additional cost. Therefore atlas-based approaches are similar to classify 
methods, except that they are implemented in spatial domain rather than in the feature space. 

The alignment of a probabilistic atlas with the image to be segmented, will ensure the prior knowledge of the 
method. Since the segmentation labels and the ground truth are known for the atlas, all atlas information is transferred to 
the target image after registration. Therefore, the performance of atlas-based methods are directly dependent on the quality 
of the registration method used. 

However the brain extraction using a probabilistic atlas is usually not very accurate and can cause 
misclassification around the brain boundary. The common way of aligning the probabilistic atlas with the image is to use 
affine registration. However if brain anatomy of interest differs from the probabilistic atlas an affine alignment may not be 
sufficient. Therefore suggest aligning the atlas using non rigid registration. But in other literature shows difficulties in 
registering anatomical template with the image to be segmented using standard registration methods. In addition, even with 
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non-rigid registration methods, accurate segmentation of a complex structure is difficult due to anatomical variability. 

Atlas guided segmentation in patients with brain deformations can be difficult and prone to errors, because the 
probabilistic atlas is based on a population of healthy subjects. Regardless of the type of atlas-based segmentation 
(single or multi) or the atlases used, the accuracy of segmentation is highly dependent on the registration methods utilized 
[11, 12, 13, 14], 

Markov Random Field Models 

MRF modelling itself is not a segmentation method, but a statistical model that can be used within segmentation 
methods. MRFs model spatial interactions between neighbouring or nearby pixels. These local correlations provide a 
mechanism for modelling a variety of image properties (60). 

In medical imaging, they are typically used because most pixels belong to the same class as their neighbouring 
pixels. In physical terms, this implies that any anatomical structure that consists of only one pixel has a very low 
probability of occurring under an MRF assumption. MRFs are often incorporated into clustering segmentation algorithms 
such as the K -means algorithm under a Bayesian prior model. 

The segmentation is then obtained by maximizing the a posteriori probability of the segmentation, given the 
image data. This maximization can be achieved by iterative methods such as iterated conditional modes or simulated 
annealing. A difficulty associated with MRF models is proper selection of the parameters controlling the strength of spatial 
interactions. A setting that is too high can result in an excessively smooth segmentation and a loss of important structural 
details. In addition to that, MRF methods usually require computationally intensive algorithms. Despite these 
disadvantages, MRFs are widely used not only to model segmentation classes, but also to model intensity in homogeneities 
that can occur in MR images and textural properties, which is useful in the segmentation of digital mammograms 
[15, 17, 18, 19, 20]. 

Region Growing Method 

Region growing is a technique for extracting a connected region of the image which consists of groups of 
pixels/voxels with similar intensities. In its simplest form, region growing starts with a seed point (pixel/voxel) that 
belongs to the object of interest. The seed point can be manually selected by an operator or automatically initialized with a 
seed finding algorithm. Then, region growing examines all neighboring pixels/voxels and if their intensities are similar 
enough (satisfying a predefined uniformity or homogeneity criterion), they are added to the growing region. 

This procedure is repeated until no more pixels/voxels can be added to the region. Region growing is suitable for 
segmentation of volumetric images which are composed of large connected homogeneous regions. Thus, it is successfully 
used in medical image analysis to segment different tissues, organs, or lesions from MR images. For example, it is used in 
the brain MRI analysis for segmentation of brain vessels, brain tumour segmentation, or extraction of brain surface. 

The primary disadvantage of region growing is that it requires manual interaction to obtain the seed point. Thus, 
for each region that needs to be extracted, a seed must be planted. Split and-merge is an algorithm related to region 
growing, but it does not require a seed point. Region growing can also be sensitive to noise, causing extracted regions to 
have holes or even become disconnected. Conversely, partial volume effects can cause separate regions to become 
connected. To help alleviate these problems, a homo-topic region-growing algorithm has been proposed that preserves the 


Impact Factor (JCC): 7.9878 


NAAS Rating: 3.19 



Eyeing the Human Brain’s Segmentation Methods 


43 


topology between an initial region and an extracted region [21, 22, 23, 24, 25]. 

Convolution Neural Network 

In recent years, a growing interest has been drawn to the Convolution Neural Network. Having a remarkable 
performance in classification and detection, researchers seem to extend its use in the segmentation tasks. The need of 
alternative approach arises due to the fact that the images accumulated from imaging modalities depict divergent 
morphological patterns of the brain structures. The rationale is linked to variations in imaging parameter settings. 

Therefore, Deep learning is said to be effective in exploiting spatial structures of images by utilizing local 
features. Through feature descriptors, Deep learning provides distinct characteristics potential for segmentation unlike the 
previous methods. A typical example of CCN is given in fig. 3. Other researchers argue that CNN capability exceeds 
human imagination as it is believed that even lowest contrast pathological features can be captured by this approach; 
making it suggested for many clinical applications. 

Although the method becomes a future hope, there are few challenges to be underlined associated to CNN lone, 
CNN depends on massive data to promise a good result which seem to be a problem clinically because most of clinical 
cases consist of limited data set unlike common natural images; second, Medical images exhibit 3D high resolution, 
seeking models with high dimension when dealing with them contrary to natural images. 

Nevertheless, a good news in computer vision technology inform us about advancement in Graphic Processing 
Unit, which accelerates computing speed and thus, allowing fast training of data on a complex deep networks while 
reducing top 5 error(from 26.2% to 15.3%). 

Through several trials, CNN has been proven to provide optimal solution in segmentation tasks with efficiency to 
handle bias and variance (Over fitting and under fitting). Along with such success, more other fields present magnificent 
achievements with deep learning: object detection and disease classification. Due to the adaptive utilization of deep 
learning, different form/models of DL exist, such as deep neural networks, stacked auto-encoders, deep Boltzmann 
machines, and convolution neural networks (CNNs) [ 26, 27, 28, 29, 31, 32]. 

CONCLUSIONS 

Among the aforementioned methods, CNN gives more promising result, and is a significantly powerful tool for 
segmentation. The future direction will be totally dependent of the CNN-based approaches. 

The intensity based methods gain no confidence because several regions of the brain consist of similar intensities, 
making it difficult for segmentation of the regions. The deformation-based morphometry methods fail to capture the 
intermediate range of features since the Jacobian determinant does not describe the value between the ranges; they 
threshold the values and leave other intermediate values unaccountable. 

The Multi-Atlas Registration Based Methods are currently working better on the field, but suffer from parameter 
estimations and criteria. Many data are reported to be discarded because they give bad segmentation results. 

The Markov Random Field Models provide approximately better results, but involves several approximations and 
assumptions. The region growing methods are efficient in other applications, but not useful in medical images since can 
over estimate or underestimate the pathology during the process. In this, regard, despite the challenging of a large dataset, 
CNN-based methods remain the future hope for proper segmentation with account to many challenging pinpointed in other 
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methods. The CCN-based methods are robust to brain structural changes and motion. 
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